97 research outputs found
Statistical Analysis of Dynamic Actions
Real-world action recognition applications require the development of systems which are fast, can handle a large variety of actions without a priori knowledge of the type of actions, need a minimal number of parameters, and necessitate as short as possible learning stage. In this paper, we suggest such an approach. We regard dynamic activities as long-term temporal objects, which are characterized by spatio-temporal features at multiple temporal scales. Based on this, we design a simple statistical distance measure between video sequences which captures the similarities in their behavioral content. This measure is nonparametric and can thus handle a wide range of complex dynamic actions. Having a behavior-based distance measure between sequences, we use it for a variety of tasks, including: video indexing, temporal segmentation, and action-based video clustering. These tasks are performed without prior knowledge of the types of actions, their models, or their temporal extents
"Zero-Shot" Super-Resolution using Deep Internal Learning
Deep Learning has led to a dramatic leap in Super-Resolution (SR) performance
in the past few years. However, being supervised, these SR methods are
restricted to specific training data, where the acquisition of the
low-resolution (LR) images from their high-resolution (HR) counterparts is
predetermined (e.g., bicubic downscaling), without any distracting artifacts
(e.g., sensor noise, image compression, non-ideal PSF, etc). Real LR images,
however, rarely obey these restrictions, resulting in poor SR results by SotA
(State of the Art) methods. In this paper we introduce "Zero-Shot" SR, which
exploits the power of Deep Learning, but does not rely on prior training. We
exploit the internal recurrence of information inside a single image, and train
a small image-specific CNN at test time, on examples extracted solely from the
input image itself. As such, it can adapt itself to different settings per
image. This allows to perform SR of real old photos, noisy images, biological
data, and other images where the acquisition process is unknown or non-ideal.
On such images, our method outperforms SotA CNN-based SR methods, as well as
previous unsupervised SR methods. To the best of our knowledge, this is the
first unsupervised CNN-based SR method
SinFusion: Training Diffusion Models on a Single Image or Video
Diffusion models exhibited tremendous progress in image and video generation,
exceeding GANs in quality and diversity. However, they are usually trained on
very large datasets and are not naturally adapted to manipulate a given input
image or video. In this paper we show how this can be resolved by training a
diffusion model on a single input image or video. Our image/video-specific
diffusion model (SinFusion) learns the appearance and dynamics of the single
image or video, while utilizing the conditioning capabilities of diffusion
models. It can solve a wide array of image/video-specific manipulation tasks.
In particular, our model can learn from few frames the motion and dynamics of a
single input video. It can then generate diverse new video samples of the same
dynamic scene, extrapolate short videos into long ones (both forward and
backward in time) and perform video upsampling. When trained on a single image,
our model shows comparable performance and capabilities to previous
single-image models in various image manipulation tasks.Comment: Project Page: https://yanivnik.github.io/sinfusio
Revealing and modifying non-local variations in a single image
We present an algorithm for automatically detecting and visualizing small non-local variations between repeating structures in a single image. Our method allows to automatically correct these variations, thus producing an 'idealized' version of the image in which the resemblance between recurring structures is stronger. Alternatively, it can be used to magnify these variations, thus producing an exaggerated image which highlights the various variations that are difficult to spot in the input image. We formulate the estimation of deviations from perfect recurrence as a general optimization problem, and demonstrate it in the particular cases of geometric deformations and color variations.Israel Science Foundation (Grant 931/14)Shell Researc
- …